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A METHOD AND SYSTEM FOR UTILIZING A SELF-SIMILARITY 
TECHNIQUE TO PROCESS AN IMAGE 

FIELD OF THE INVENTION 

The present invention relates generally to image scanners and more particularly 
to a method and system for utilizing a self-similarity technique to process an image. 



BACKGROUND OF THE INVENTION 

Image processors (scanners, copiers, fax machines, etc) convert a visible 
image on a document or photograph, or an image in a transparent medium, into an 
electronic form suitable for copying, storing or processing by a computer. An image 
processor may be a separate device or may be a part of a copier, part of a facsimile 
machine, or part of a multipurpose device. In general, image processors use an 
optical lens system or optical waveguide system to focus an image onto an array of 
photo sensors. In some processors, a Gradient Index Lens Array (GILA) is employed. 
A GILA is a row of lenses that goes across the width of the page being processed. 
One of the issues with the utilization of a GILA is that the farther out of focus the 
image moves the more the image has a tendency to be seen by multiple lenses. This 
causes the creation of aberrations known as "echoes" or the "echo effect". Because 
the array of lenses is in a row, the echo effect is one-dimensional in nature, occurring 
only in the horizontal direction. As a result of the echo effect, a "de-echoing" process 
is desirable in order to minimize the presence of these aberrations or echoes. 

De-echoing belongs to the class of de-convolution tasks. In this case the 
convolution that has to be undone is assumed to have a train of pulses as a kernel. 
These pulses usually have varying intensities and they occur a fixed distance d from 
neighboring pulses. Since the out-of-focus distance is known, the convolution kernel 
is known. 

For a better understanding, please refer to Figure 1 A - Figure ID. In each 
Figure, the vertical axis represents an intensity valuer and the horizontal axis 
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represents a number of pixels d. Figure 1 A shows a 1 dimensional input signal 
(image) 100. Figure IB shows a train of pulses 1 10 with variable amplitudes. In this 
particular example the distance between pulses is d = 6. Figure 1C shows a train of 
echoes 120 which are the result of convolving the input signal 100 with the train of 

5 pulses 110. 

It should also be noted that when an input signal is convolved with a train of 
pulses where d = 1, the resulting signal is not referred to as a train of echoes but rather 
a blurred version of the input signal (image). Likewise, the "train of pulses" is not 
referred to as such, but rather as a blurring kernel. Figure ID shows a resulting signal 

10 130 when the input signal is convolved with a train of pulses with d = 1 . In either 

case (de-echoing or de-blurring), the convolution kernel is known and de-convolution 
is needed in order to minimize the presence of these aberrations. 

A conventional solution for the de-convolution of a known kernel is to divide 
the Fourier coefficients of the measured signal by the Fourier coefficient of the kernel 

1 5 and apply an inverse Fourier transform. However, this solution is unstable in the 

presence of noise. Other techniques involve the implementation of iterative 
algorithms that require substantial computation complexity. 

In general, what is needed is a method and system that addresses the above- 
referenced problems associated with the de-convolution of an input image. Additionally, 

20 the method and system should be simple, inexpensive and capable of being easily 

adapted to existing technology. The present invention addresses these needs. 
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SUMMARY OF THE INVENTION 

A method and system for utilizing a self-similarity technique to process an 
image is disclosed. In accordance with embodiments of the present invention, a 
processing algorithm is generated based on a self-similarity assumption. Through the 
use of the method and apparatus in accordance with the present invention, imperfections 
that are present in a degraded image can be compensated for in a simple, fast and non- 
iterative fashion thereby resulting in a higher quality image. 

A first aspect of the present invention is a method for utilizing a self-similarity 
technique to process an image. The method includes obtaining a corrupted image, 
altering the corrupted image to obtain an altered image, determining a plurality of 
parameters of a parametric mapping operator for mapping the altered image into the 
corrupted image and utilizing the plurality of parameters to map the corrupted image into 
an output image. 

A second aspect of the invention is a system for utilizing a self-similarity 
technique to process an image. The system includes means for obtaining a corrupted 
image, means for altering the corrupted image to obtain an altered image, means for 
determining a plurality of parameters of a parametric mapping operator for mapping the 
altered image into the corrupted image and means for utilizing the plurality of parameters 
to map the corrupted image into an output image. 

A third aspect of the present invention is a scanning apparatus. The scanning 
apparatus includes a processor, an operating system coupled to the processor and a 
scanning module coupled to the operating system wherein the scanning module 
comprises logic for instructing the processor to perform the steps of obtaining a 
corrupted image, altering the corrupted image to obtain an altered image, determining a 
plurality of parameters of a parametric mapping operator for mapping the altered image 
into the corrupted image and utilizing the plurality of parameters to map the corrupted 
image into an output image. 

Other aspects and advantages of the present invention will become apparent from 
the following detailed description, taken in conjunction with the accompanying 
drawings, illustrating by way of example the principles of the invention. 
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BRIEF DESCRIPTION OF THE DRAWINGS 



Figure 1 A shows a 1 dimensional input signal. 

Figure IB shows a train of pulses with variable amplitudes. 
5 Figure 1 C shows a train of echoes which are the result of convolving the input 

signal with the train of pulses. 

Figure ID shows a resulting signal when the input signal is convolved with a 
train of pulses with d = 1 . 

Figure 2 is a high-level flow chart of a method in accordance with an 
1 0 embodiment of the present invention. 

Figure 3 shows a system for performing a method in accordance with an 
embodiment of the present invention. 

Figure 4 shows a tuning table in accordance with an embodiment of the present 
invention. 



15 
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DETAILED DESCRIPTION 

The present invention relates to a method and system for processing an image. 
The following description is presented to enable one of ordinary skill in the art to make 
and use the invention and is provided in the context of a patent application and its 
requirements. Various modifications to the embodiments and the generic principles and 
features described herein will be readily apparent to those skilled in the art. Thus, the 
present invention is not intended to be limited to the embodiment shown but is to be 
accorded the widest scope consistent with the principles and features described herein. 

A method and system for processing an image is disclosed. In accordance with 
embodiments of the present invention, a processing algorithm is generated based on a 
self-similarity assumption. Through the use of the method and apparatus in accordance 
with the present invention, imperfections that are present in a degraded image can be 
compensated for in a simple, fast and non-iterative fashion thereby resulting in a higher 
quality image. For the purposes of this patent application, an image can be textual, 
graphical, digital, analog or a variety of other types while remaining within the spirit and 
scope of the present invention. 

For a better understanding of the present invention please refer to Figure 2. 
Figure 2 is a high-level flow chart of a method in accordance with an embodiment of the 
present invention. A first step 210 includes obtaining a corrupted image. In varying 
embodiments, the corrupted image is an echoed or blurred image. A second step 220 
includes altering the corrupted image to obtain an altered image. In varying 
embodiments, altering the corrupted image includes producing an echoed or blurred 
version of the corrupted image. For example, if the corrupted image includes echoes, 
altering the corrupted image involves echoing the echoed image. Similarly, if the 
corrupted image is a blurred image, altering the corrupted image involves blurring the 
blurred image. 

A third step 230 involves determining a plurality of parameters of a parametric 
mapping operator for mapping the altered image into the corrupted image. A final step 
240 involves utilizing the plurality of parameters to map the corrupted image into an 
output image. 
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The above-described embodiment of the invention may also be implemented, for 
example, by operating a computer system to execute a sequence of machine-readable 
instructions. The instructions may reside in various types of computer readable media. 
In this respect, another aspect of the present invention concerns a programmed product, 
comprising computer readable media tangibly embodying a program of machine- 
readable instructions executable by a digital data processor to perform the method in 
accordance with an embodiment of the present invention. 

This computer readable media may comprise, for example, RAM contained 
within the system. Alternatively, the instructions may be contained in another computer 
readable media such as a magnetic data storage diskette and directly or indirectly 
accessed by the computer system. Whether contained in the computer system or 
elsewhere, the instructions may be stored on a variety of machine readable storage 
media, such as a DASD storage (for example, a conventional "hard drive" or a RAID 
array), magnetic tape, electronic read-only memory, an optical storage device (for 
example, CD ROM, WORM, DVD, digital optical tape), or other suitable computer 
readable media including transmission media such as digital, analog, and wireless 
communication links. In an illustrative embodiment of the invention, the machine- 
readable instructions may comprise lines of compiled C, C++, or similar language code 
commonly used by those skilled in the programming for this type of application arts. 

Figure 3 shows an example of a system 300 for performing a method in 
accordance with an embodiment of the present invention. In this particular embodiment, 
the system 300 is a scanner or scanning apparatus. The system 300 includes a control 
circuit 314 for controlling operations of the system 300 and a scanning module 316 
connected to the control circuit 314 wherein the scanning module includes logic for 
performing the method in accordance with the present invention. The system 300 further 
includes a user interface 320 connected to the control circuit 314 wherein the user 
interface 320 includes a plurality of keys for inputting key signals. 

The control circuit 314 includes a memory 324 for storing programs and data, 
and a processor 326 for executing the programs in the memory 324. The memory 324 
includes an operating system 328 for controlling the operations of the processor 326, a 
scanning module driving program 330 executed under the operating system 328 for 
controlling the operations of the scanning module 316 and processing the document 
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image signals generated by the scanning module 3 16, a user interface driving program 
332 executed under the operating system 328 for controlling the operations of the user 
interface 320, and an application program 336 executed under the operating system 328 
for setting the operation mode of the scanning module driving program 330 and the 
processing method of the image signals according to the key signals inputted by the user 
and transmitted by using the user interface driving program 332. 

As previously mentioned, the present invention employs self-similarity 
techniques to process an image. Self-similarity involves the notion that natural 
images are self-similar or in other words there is a high correlation between different 
scaling levels of natural images. 

This observation leads to the following paradigm for image scaling. In order 
to upscale an image, first downscale it i.e. in order to undo an image degradation, an 
initial step involves a further degradation of the image. 

Although this paradigm is described in the context of image scaling, the 
paradigm can be extended to other image processing tasks as well. For example, if an 
image needs to be de-echoed, first echo the image; if an image needs to be de-blurred, 
first blur the image, etc. Next, find the parameters of a parametric mapping operator 
that brings the further echoed or blurred image as close as possible to the original 
echoed or blurred image. Finally, the same parameters are utilized to map the original 
echoed or blurred image into an output image. 

Although the above-described embodiments of the present invention are 
disclosed in the context of a de-echoing or a de-blurring process, one of ordinary skill 
in the art will readily recognize that the process could be applied to a variety of 
different types of image processing tasks while remaining within the spirit and scope 
of the present invention. 

The following is an illustration of some mathematical relationships that 
characterize a self-similarity approach in accordance with an embodiment of the 
present invention. In this illustration, a 1 -dimensional measured signal g(x) is 
obtained by a linear translation-invariant degradation of an ideal signal f(x) by a 
kernel k(x) as shown in Eq. (1); 



g(x) =f(x) * k(x) 



(1) 
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What is accordingly desired is a determination of an approximation, F(x), of the 
underlying signal f(x). 

In a self-similarity approach, a first step towards determining F(x) is to further 
degrade the measured signal, g(x) by the same kernel k to obtain a filtered signal h(x) 
as shown in Eq. (2): 

h(x)=g(x)*k(x) (2) 

Next, a parametric mapping operator M p i is assumed and a set of parameters p that 
maps the filtered signal h to "a good" approximation G of the measured signal g is 
found as shown in Eq. (3): 

G(x)=M pl {h(x)J (3) 

Finally, once p is determined, M pi is used to map the measured signal g to an 
approximation signal F of /as shown in Eq. (4): 

F(x)=M pl {g(x)}. (4) 

Although Eqs. 1-4 depict fundamental mathematical relationships involved in 
a self-similarity approach, an improved processing operation employs a parametric 
mapping operator, M p2 , similar to that used in fractal coding/scaling algorithms. 
Fractal scaling/coding algorithms implement mapping operators that are based on 
adaptive shifting. Consequently, the following parametric mapping operator, M p2i is 
shown in Eq. (5): 



M p2 {u} = u[x+p(x)], 



(5) 
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The parameter p(x) can then determined from the relationship between h(x) and g(x), 
by minimizing the cost function \g(x)-h(x + p(x))\. In other words, for each value of jc, 
the parameter p(x) is that value for which the function ]g(x)-h(x+p(x))\ achieves its 
minimum. 

The ideal signal approximation, F(x) 9 can then be determined as follows: 

F(x) =M p2 {g} =gfx +p(x)J. (8) 

This solution, however, does not have the desired effect of boosting the 
contrast of the image. An improved process is shown in Eq. 9 for a parametric 
mapping operator M p s\ 

M p3 fu} = u(x) +ufx +p(x)J- (u*k) fx +p(x)J, (9) 

where p(x) is determined as described above. The approximation signal is 
then given by: 

F(x) = M p3 {g} = g(x)+ gfx +p(x)J -hfx + p(x)J. (10) 

In an embodiment of the present invention, a discrete signal f(i) is employed, 
where i is an integer index. Here, for each value of /, the mapping parameter p(i) is equal 
to the integer n € TV, for which the cost function \g(i)-h(i+n)\ achieves its minimum. 

In order to reduce computations, the search for optimal displacement n is 
limited to only three values: -d, 0 and </, where d is the distance between echoes. In 
order to compensate for the loss of quality caused by the limitation of the search 
range, fuzzy mapping can be implemented instead of Equation (10) whereby the 
approximation signal, F(x)> can be determined as follows: 

F(i)=g(i) + A. Z a„[g(i + n)-h(i + n)], (11) 

n6{-d,0,d} 
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where A is a strength parameter. 

The parameters On are accordingly obtained as a function of the relative 
similarity between g(i) and h(i+n). More specifically, calculate the similarity values 
Sn = \g(i) - h(i+n) I for all the three values of «, and the minimum of all three 
similarities m = min(s^ s 0y s<t). The relative similarity r„ 9 for each value of «, is given 
by r„ = s n -m 9 which is always non-negative. 

Now calculate On-tpfr^, wherep is a pre-determined function. In one 
embodiment, <p(x) is as follows: 

q> (x) = ml n (1, T/(x+l)), (12) 

where J is a tunable parameter. 

In general, the function p returns high values for small input values, and vice- 
versa. Finally, the parameters On are obtained by normalizing the values a„\ i.e., cc 

The strength parameter A and the pulse distance d can be manually tuned for a 
fixed output resolution of the approximation signal. However, A and d should vary as 
a function of the output resolution as shown in Figure 4. Figure 4 shows a tuning 
table 400 in accordance with an embodiment of the present invention. The tuning 
table 400 includes a resolution column 410 and parameter columns 420-440. 
Resolution column 410 includes a variety of different resolution values and parameter 
columns 420-440 illustrate empirically obtained X and d parameter values. 

For example, suppose d = d r = T (for some integer ri) and X - Xr at a given 
resolution r. If n > 0, then at a resolution r/2, X is kept the same, whereas d is set to 
d r 12 = 2 (n ' l K If, however, n = 0 (i.e. d r = 1) then, at resolution r/2, d is kept equal to 1 , 
whereas X is set to A/2. This rule should be iterated for resolutions r/4, r/4, r/8 and so 
forth. 

For instance, suppose for a certain scanning device, d = 4 and A =2 at the 
resolution of 2400 ppi (pixels per inch). Then, the values of d and A for other 
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resolutions can be set as shown in parameter column 440. 

Based on the above outlined mathematical relationships, a processing 
algorithm in accordance with an embodiment of the present invention can be 
characterized as follows: 
5 Given a degraded 2D discrete image G(iJ), do the following for each line g/i) 

of G(i J): 

1 . Calculate a further degraded version of the line: 

10 hj (i) = ( gj * k)(i), (13) 

where k(i) represents the echoing kernel, and includes a pulse train with a distance d 
between the neighboring pulses. In an embodiment, intensities k(-2d) = k(2d) = 
0.138, k(-d) = k(d)= 0.222, and k(0) = 0.28. These intensities do not need to be fixed 
1 5 and could be updated for different image processing devices. 

2. For each pixel i, calculate the similarity values: 

s. d =/gj(i)-hj(i-d)/, (14) 
20 s 0 = /gjd)-hj(i)/, (15) 

s d =/gj(i)-hj(i+d)/, (16) 

and the associated minimum, m - min(s^ s 0 , s<t). In an embodiment, the distance d 
between pulses is obtained, for example, from the image processing device 
25 manufacturer and could depend on a variety of factors related to the image processing 

device including but not limited to an out-of- focus distance, an acquisition resolution, 
an optical system characteristic, etc. For example, referring back to Figure 4, Figure 4 
shows an example of a tuning table 400 where d is a function of the resolution r. 

30 

3. Use these values to calculate the non-normalized a mapping parameters as 
follows: 
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a. d = min (l,T/s^ - m+1)), 
ao'= min (l,T/s 0 - m+1)), 
aj = min (1, T/s d - m+1)), 



(17) 
(18) 
(19) 



In an embodiment, a value of J=3 is utilized to obtain good results, however one of 
ordinary skill in the art will readily recognize that a variety of different values of T 
could be utilized while remaining within the spirit and scope of the present invention. 

4. The normalized a mapping parameters are then given by: 



5. Calculate the output value O(ij) of the pixel utilizing the normalized mapping 
parameters: 



where X is the strength parameter. Accordingly, Eq. (23) has 5 parameters, 
two inputs, and one output. The two inputs are the signals g and h and the output is 
the image O. Among the 5 parameters, there are 2 global parameters [ i.e., parameters 
that do not depend on the pixel position (i j)] f and 3 local parameters [parameters that 
are recalculated for each pixel (i j)]. The global parameters are d and X and the local 
parameters are the Ct parametric mapping operator parameters. 

In an embodiment, step 1 is performed for each line only once, by calculating 
the values of h/i) for all possible values of i before proceeding to step 2. Steps 2 
through 5 are then performed iteratively for each pixel L In an alternate embodiment, 
steps 1-5 are performed iteratively for each value of i, where the calculation of hj in 



OLd = OLd/ (cLd+Oo'+CCd), 

ao = cti/(cLd+ao+ad), 
ad —OLd / (oLd ^oco+ad)y 



(20) 
(21) 
(22) 



O(iJ) = g/i) + X{0L d f gj (i-d) - hj (i-d)]+ 

ao[gj(i)-hj(i)] + 
a d [ gj (i+d)-hj(i+d)]}, 



(23) 
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step 1 is done at each iteration for the pixels /-rf, i, and i+rf only. 

Although the above-described implementation involves a two-dimensional 
image, each line is addressed separately. Therefore, the embodiment of the proposed 
process is one-dimensional in nature. One of ordinary skill will readily recognize that 
a two-dimensional extension of the process is fairly straightforward and within the 
spirit and scope of the present invention. 

Additionally, the above-described image processing technique is implemented 
based on a monochromatic input signal. However, in dealing with a color image, a 
variety of different implementations can be employed. In a first implementation, the 
algorithm can be applied to each color plane separately. In a second implementation, 
the color image is transformed from the input Red Green Blue(RGB) space to a color 
space where one of the components is a luminance component (YIQ, YcbCr, Lab, 
etc.). The luminance channel is processed with the above-described technique and the 
result output RGB space is transformed. A final disclosed implementation, processes 
only the Green channel of the input signal with the above-described technique and 
computes the Red and Blue channels as follows: 

F R (ij) = G R (i j) + D(iJ) (24) 
F B (iJ) = G B (i j) + D(iJ) (25) 

where F Rf F& and F B are the red, green, and blue channels of the output image, 
respectively, Gr, G& and Gb are the red, green and blue channels of the input image, 
respectively, and 

D(iJ) = FodJ) - GadJ), (26) 

A method and system for processing an image is disclosed. In accordance 
with embodiments of the present invention, an image is processed utilizing an image 
processing technique based on a self-similarity assumption. Through the use of the 
method and apparatus in accordance with the present invention, imperfections that are 
present in a degraded image can be compensated for in a simple, fast and non-iterative 
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fashion thereby resulting in a higher quality image. 

Although the present invention has been described in accordance with the 
embodiments shown, one of ordinary skill in the art will readily recognize that there 
could be variations to the embodiments and those variations would be within the spirit 
5 and scope of the present invention. Accordingly, many modifications may be made by 

one of ordinary skill in the art without departing from the spirit and scope of the 
appended claims. 



